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PROMOTING GROWTH IN PROBLEM SOLVING IN AN INTEGRATED 
PROGRAM OF LANGUAGE SKILLS FOR THE FIFTH GRADE 



Abstract 



In an experiment a battery of pretests and posttnsts was developed 
and administered as part of a 23-week program in the cultivation of cog- 
nitive skills. Results obtained from 200 Ss scores in a total of 24 
classes, 6 classes in each of 4 treatment groups, were significant for 
overall tests (£ «^.01) and for planned contrasts in favor of two ex- 
perimental groups: (a) a group receiving the basic treatment in prob- 

lem solving plus extra training in abstract thinking and (b) a group 
receiving the basic treatment plus the extra practice in problem solving. 
A supplementary analysis of high, medium, and low IQ levels indicated 
similar results. 
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PROMOTING GROWTH IN PROBLEM SOLVING IN AN INTEGRATED 
PROGRAM OF LANGUAGE SKILLS FOR THE FIFTH GRADE 1 



S. W. Lundsteen 
University of Texas at Austin 



Problem solving has been used to refer to such a wide diversity 
of behavior that it is a poorly defined, although frequently stated, 
goal of elementary education. Creative behavior in problem solving 
has been described by various authorities, for example, Guilford (1967), 
Gctzels (1964), and Russell (1956, 1965). The creative aspect is dis- 
tinguished by Guilford as a divergent dimcnsion--a way of generalizing 
information from given data. In a divergent thinking sequence individuals 
are free to generate Independently their own data within a data-poor 
situation. Individuals may take a new direction or gain a new perspective 

In agreement, Getzels has defined the creative dimension in problem 
solving as related to the number of knowns and unknowns in the situa- 
tion. He suggests strongly that school problems be full of unknowns 
both as to the problem and as to the solution, at least for the pupil. 

He suggests that too often in school there are problem situations and 
measurement where both pupil and teacher know what the problem is and 
what the one right answer is. Thus, the pupil behavior is a matter of 
rote drill. 

Russell (1956, 1965) distinguished six basic mental processes: 

(a) perceptual thinking, (b) associative thinking, (c) inductive- 
deductive thinking leading to concept formation, (d) creative thinking, 

(e) critical thinking, and (f) problem solving. It appeared that the 
other five processes might be found Interrelated meaningfully during 
problem solving which was organized with high consciousness and orienta- 
tion toward a goal perceived by the child. But perhaps the best and 
most recent exposition of the "problem with 'problem solving'" is pre- 
sented in a review by Kelslar (1969) for the school setting. 

The present investigator could find no study that had tried to 
construct tests designed to evaluate a program to cultivate a problem- 
solving process including a creative dimension within a program of 
integrated language skills in the elementary school. As well as being 
academically oriented, the present program was Intended to be "people 
oriented" so that the portion of the problem-solving process dealing 
with application of principles (Bloom, 1956) dealt with human rela- 
tions that might be used by children. The integrated language skills 
program is described in detail elsewhere (Lundsteen, 1968). 



The author acknowledges the support of the Charles F. Kettering 
Foundation in this investigation, and statistical analysis donated by 
the Research and Development Center, The University of Texas «*. Austin. 
The author is grateful to Professor Bepjamin Fruchter for his sub- 
stantial aid in criticising this report before submission. 
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Accordingly, the major concern of this study was to assist and to 
evaluate development of pupil behaviors in the process of creative prob- 
lem solving. Integrated into the rationale and behavioral objectives 
were opinions from the previously cited authors and others (Rickbom 
and Lundstccn, 1968). The terminal behavior of creative problem solving 
involved 12 steps, ordered flexibly, and 7 qualities. The qualities 
were incorporated into a judgmental scale. An experimental study was 
conducted in order to manipulate training for the assistance of the 
problem-solving process and two related subabilitics: (a) skill in 
listening and (b) skill in employing an abstract level of thinking. In 
a gross way an attempt was made to deal with learning hierarchies accord- 
ing to the theory suggested by GagnCf (1966). Parenthetically, Gagi.d has 
used the tern "subcapability" to label a skill which is subsidiary to 
some terminal behavior. In this report the term is shortened to "subability." 

The first task in this investigation was to see if there was any 
significant difference between the experimental groups that received 
the problem-solving training and the control groups that did not. The 
controls followed the state prescribed curriculum for Santa Barbara 
County, California. A second task was to determine what each of the 
three experimental treatments might contribute. The experimenter pre- 
dicted that the treatment including not only problem solving but also 
abstract thinking would have the highest payoff, probably even higher 
than the group assisted with listening skills. See Table 1 for a 
representation of the four treatments. The stimulus for the Wednesday 
problem-solving treatment was presented via television. The second 
control group had postests only. 



Method 



Subjects 

From a pool of 45 volunteer classes, 6 were assigned randomly to 
each of the 5 treatment groups. For the Judged battery of three tests 
of problem solving, a subsample was selected randomly from each class. 
In the final analysis N equalled 200 Ss. (See Table l for the indica- 
tion of N in each group.) 

Instruments 



In this section is presented each of three criterion measures of 
problem solving and information on (a) how the student reported his 
knowledge, (b) the nature of the question asked or the "display," 

(c) the judgmental continuum or scale used to assess the quality of 
performance during the profile of the 12-step sequence, and (d) the 
method of scoring. 

For the first measure, the Oral Composition Task of Problem Solv- 
i i (OCOM)t the student reported by speaking into a microphone, The 
E used small transistor tape recorders which children experienced for 
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Basic Elements for Treatment Groups 
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several weeks before the testing. The S was asked to ”amc as many as 
three problems he might, have with a friend or with people. Next, he was 
to pick one of his problems and tell how he might go about solving it. 

It was hoped that the verbally capable child that could not write well 
would have an opportunity to display his skill. The tapes were typed 
and the responses judged. 

The fi'st chart shows the steps and a tentative instructional 
hierarchy and the second chart shows the judgmental scale of qualities 
which was applied to both the oral and the written compositions and 
also a tentative instructional hierarchy. The grouping and labeling 
step (Item 7 in the first chart) was omitted for Tasks 2 and 3. Instead, 
the number of hypotheses was counted as an index of fluency, and also the 
quality of the hypotheses was examined. With three judges, the S could ^ 
accumulate as many as 180 points.' To make as mary as two points ( fair, 
corresponding to Quality 2) on the scale, the £ simply had to construct 
a literal, relevant statement corresponding to Quality 1 in the second 
chart. For example, "Tommy's problem was that he messed up his neigh- 
bor's patio," would be classified as representing Quality 1. Tcst- 
rctest reliability was .52 (N * 27), and the mean interrater reliability 
was .88 for three judges (N » 200). A total rating as high as 180 was 
possible for both this measure and the second measure. 

For the second measure, the Written Composition Task of Problem 
Solving (WRIC), the S reported his skill by writing on paper. Other- 
wise, this task was exactly the same as the first and was judged using 
the same scale. Test-retest reliability was .53 (N * 26), and the 
mean interrater reliability was .87 (N * 200). 

For the third measure (the open stories) called Tell the Problem 
and Make a Plan (OPST) , the S again reported his knowledge by writing. But 
he was stimulated by an unfinished story, rather than having to furnish 
his own data. Also, he was given 12 stimulus questions to aid in his con- 
struction of responses to the 12 steps in problem solving described 
earlier. Test-retest reliability was .74 (N « 26), and the mean inter- 
rater reliability was .90 (N * 200). 

The stories were all about boys who had problems such as relating 
to their peers, coping with feelings of guilt, with poor self-picture, 
and with living up to parents' expectations. There were 24 items in all, 
with two chances to respond to each type of question. With three judges, 
the S could accumulate as many as 1,224 points, with generally 2 points 
for each question and for each category. 

The rating criteria were stated in performance language. (See Chart 2.) 
Names for the objectives Included: (a) Comprehension, (b) Relevance, (c) 

Elaboration, (d) Originality, v e) Multiple Alternatives, (f) Empathy, 

(g) Causal Thinking, (h) Testability of Hypotheses, and (i) Evaluation. 

Thus, the S was measured not only on a pass-fail basis (i.c., Did ho 
respond to*"the step or not respond to the step?), but also attempts 
were made to assess a hierarchy of qualities concerning the responses, 

Also an analysis of difficulty level of the 1/ question types has been 
made for this measure. (See Table 6.) 
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Chart 1 

TWELVE STEPS OR QUESTION TYPES USED IN THE TESTS 
FOR CREATIVE PROBLFM SOLVING 



l Main Problem * The pupil constructs a problem. 

2. Subproblems . The pupil names or constructs subproblems that go with 
the main problem or are a part of it. 

3. Definition , The pupil defines (describes) key terms in the problem 
situation which he constructed and/or clarifies these terms. 

4. Type of problem . The pupil identifies (distinguishes) the problem 
as to type or kind . 

5. Hypotheses . The pupil constructs hypotheses. 

6. Facts and conditions . The pupil names, describes, and distinguishes 
facts and conditions in the situation which he constructs that might 
be helpful when seeking a solution to the problem. 

7. C lassification of facts and conditions . The pupil distinguishes, 
groups or classes and names the class(es) under which the various 
facts and conditions might be categorized. 

8. Hi ssing data . The pupil names and/or describes and/or distinguishes 
missing information that is needed to solve the problem. 

9. Search strategy . The pupil constructs a method(s) for finding the 
information that is missing. 

10. Statement of principle . The pupil states a principle for solving a 
problem of the type he indicated. 

lli Application of principle . The pupil applies the principle to the 
problem situation, describing the application, 

]2. Planned evaluation of hypothesis . The pupil identifies one of his 
constructed hypotheses for solving the problem and constructs an 
evaluation by supporting his choice with reasons in regaru to con* 
fidence in probability and/or consequences. 
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Block Diagram 



TENTATIVE LEARNING HIERARCHY FOR THE INSTRUCTIONAL SEQUENCE IN 

PROBLEM-SOLVING BEHAVIORS 
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Chart 1 

QUALITIES OR TRAITS SOUGHT FOR RFSPONSES DURING PROBLEM SOLVING 
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1. Li tern 1 Comp r o h c n s i cm (Prerequisite): The child identifies, names, 

describes and/or constructs a response shewing plain, literal com- 
prehension of specifics. (Reference: Bloom, 1956, Taxonomy of Ed * 

ucational O bj ectives , Cognitive Dorjain Level 1.) Note: Initially 

the instructor has the responsibility to determine if the stimulus, 
c.g., story episode , is suf f iclc.it iy relevant to the child so that 
literal comprehension is possible. Later, the child has part res- 
ponsibility for transforming unfamiliar problems into the familiar.) 

2. Relevance : The child identi fies .. .construe ts a response showing a 

connection to the stimulus or story, or that he has "stuck to the 
top.c" rather than "Roing off into the wild blue yonder," not to 
return. (Note: The instructor is cautioned, however not to shut 

off thinking, especially of the timid child, when guiding the dis- 
cipline of relevance necessary to & goal oriented task. Again, 

in the first place the goal needs to be relevant to the child, 
o.g., the child needs to sense a problem. (Bloom, 1956, Taxonomy 
level 1) 

5. E laboi at ton : The child Identifies. . .construe ts a response with 

details, cxanplcs, and descriptions which indicates fluency jnd 
which amplifies or enlarges upon the step in the problem-solving 
process. The response may nave been stimulated by a question 
from the instructor. (Bloom, 1956, Taxonomy Level 2). 

4, Original ity : Child identifies. . .constructs a response which is 

unique or unusual, new or different In the class or group in 
which he Is judged. Generally, others have not thought of the 
response. The response has not been one repeated by others and 
is not a limiting mental set. The response may indicate free 
reign given to speculation and fantasy. (Guilford, 1967, diver- 
gent dimension). 

5. Multiple- Altern a tives : The child ident if ics .. .constructs a res- 

ponse showing an understanding of several alternatives rather than 
Just one. His response shows understanding that goes below surface 
or literal events to inference making. The response does not repre- 
sent inflexible, rigid thinking, e.g., giving one right answer, 
using words such as "always" and "never" to make over generalisations 
or getting in a comer. (Guilford, 1967, products of Implications). 

6* Empathy : The child identifies .. .constructs a response in which he 

appears to put himself in another's position or role. He appears 
in his verbal Itatlon to show a feeling for or sympathy for someone 
else besides himself or an acceptance or preference for this value, 
(Krathvohl , 1964, Affective Domain, Level 3 Valuing) 

7. Causal Thinking : The child idcntlf ics. . .construe ts consistently a 

response showing a relationship between cause and effect. He tries 
to tell why or for vhat reason events may have happened. He gives 
an insightful explanation. Answers nay range from simple to com- 
plex, from those showing logical inadequacies to those shoving valid- 
ity. This kind of thinking frequently Is prompted after an hypo- 
thesis for solving the problem has been selected for a plan and is 
justified as to why it night work best. (Cullfotd, 1967, products 
of relations) 

6 Test ability of Hypotheses : The child idcntlf let, • .construe ts res- 

ponses for solving a problem which would be possible to assess, ob- 
serve. measure, responses which arc usable, feasible, or ptactical. 
The child produces Ideas which he could examine systematica l ly and 
critically. (Eicon, 1956, Taxonomy Level 3- -application) 

9. Evaluat len : The child Ident 1 fits .« .construe ts a response showing 

judgmental effort with a conscious criteria- -external consensus or 
internal logic--and he supports his critical thinking with reasons 
for his Judpicnt \Ailch may deal with confidence in probability ot 
consequences. Ability to give reasons is taken as evidence that 
the Judgment was not "snap" or premature but was highly conscious 
or shows "ylanfutneis. " (Bloom, 1956, level 6 • -evaluation) . 
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Block Diagram 

A TENTATIVE HIERARCHY FOR TEACHING THE QUALITIES OR TRAITS 
SOUGHT FOR RESPONSES DURING PROBLEM SOLVING 
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Procedure 



After the battery of pretests, only 3 of which are considered in 
this report, 23 weeks of instruction followed for the 4 different 
tracks, (See Table 1,) For the three experimental groups, special 
training materials were constructed and distributed to appropriate groups 
of teachers* A concept map or developmental hierarchy was made for the 
problem-solving treatment from the list of 12 steps and the 9 qualities 
sought. (See Charts on pages 5 and 7,) A portion of the hierarchy was 
distributed in each of the training lessons and repeated at planned in- 
tervals, 

Teachers were trained in the use of the experimental materials 
five days before school started, once monthly in the evening, and were 
visited by observers once monthly. Efforts were made to sec that possible 
Hawthorne effects associated with other research projects and inservice 
training were also operating for the group designated as control so as 
not to bias, relatively, results obtained from teachers using the special 
materials. The control teacher programs were reported on questionnaires. 
After pupils received the battery of posttests, analyses were made using 
the analysis of variance and Scheff^T tests. 



Results 

Table 2 shows the means and standard deviations for the throe problem- 
solving variables. Table 3 gives the analyses of variance for the three 
variables. From Table 3 it can be noted that the control group (Cl) had 
the advantage at the beginning of the experiment. The experimental 
groups were behind, but at the end they were ahead on every measure. 

There were four treatment groups which completed both pretest and 
posttests, El, E2 , E3 and Cl. A fifth treatment group, C2, had posttests 
only. There was no significant difference between Cl and C2 on any 
measure, Symbols refer to the experimental groups having (a) extra 
practice in problemsolving, (b) the listening emphasis, (c) qualitative 
levels of thinking, especially abstract, and (d) the control group 
with pretests, in that order. Simple inspection indicated that there 
was no marked departure from normal distribution for any group. Analysis 
of variance and F-test are supposedly '‘robust 11 with respect to the de- 
parture from the assumption of homogeneity of variances. According to 
inspection, differences in standard deviations were not as great for the 
posttests upon which the analyses were made. 

The hypothesis regarding comparison of treatment groups on the 
criteria for creative problem solving appeared to be generally confirmed. 
As can be seen in Table 3, differences were significant for all three of 
the problem-solving measures, The results from a supplementary analysis, 
Schcff6 tests, were significant as follows. 

WRIC . For the written composition the highest mean was that of the 
experimental treatment group with extra practice on problem solving lEl) . 
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Tabic 2 

Means and Standard Deviations for Pretests and Posttcits 
For Three Tests of Creative Problem Solving 





Experimental 


Experimental 


Experimental 


Control 1 




Treatment 1 


Treatment 2 


Treatment 3 




Test 


(N “ 


41) 


(N - 45) 


(N « 39) 


(N » 37) 




M 


SD 


M 


SD 


M SD 


M SD 


Pretest 














WRIC 


46.6 


5.4 


46.8 


4.1 


47.6 4.9 


51.5 7.6 


0C0M 


48.8 


6.3 


46.7 


5.9 


46.9 6.3 


43.0 14.2 


OPST 


180.9 


60.5 


159.6 


53.1 


136.7 62.8 


216.6 84.6 


Posttest 














WRIC 


60.3 


13.7 


55.3 


5.8 


55.4 8.0 


52.4 7.8 


OCOM 


56.0 


9.7 


55.4 


7.6 


59.5 8.2 


48.0 13.8 


OPST 


331.5 


89.2 


275.4 


94.3 


353.4 132.2 


271.4 97.7 


Note. --Abbreviated : 


: WRIC * 


written composition on a problem, OCOM a oral 


composition on a 


problem, 


OPST 


= open- 


ended story with 


12 questions. 








Table 3 






Analyses of Variance 


for Scores on 


Three Posttest Measures 






Of Creative 


Problem Solving 








Source 




df 


MS 


F 


WRIC 




Treatment 


3 


426.0 


4.9* 






Error 




158 


86.4 




OCOM 




Treatment 


3 


890.2 


'k 

8.9 






Error 




158 


99.8 




OPST 




Treatment 


3 


66827.6 


6.1* 






Error 




158 


10868.4 





Note.--N = 162 (a randomly drawn subsample from each class). Abbreviated: 
WRIC = written composition on a problem, 0C0M = oral composition on a 
problem, OPST = open-ended story with 12 questions. 



*£ < . 01 . 
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El > the listening emphasis group (E2) (j> ^.lO), El^the control (Cl) 
(p'< .01). 

OCOM. For the oral composition the highest mean was that of the 
treatment group with the emphasis on qualitative levels of thinking, 
especially abstract (£3). El > Cl <£ < .01), E2 > Cl (£ <.05), E3 > Cl 
(£ <. 01 ). 

OPST. For the open stories with 12 types of questions the highest 
mean was that of the group with the emphasis on qualitative levels of 
thinking (E3). Fl (extra practice) > Cl (£ < .10), E3 > E2 (listening) 

(£ < .05), E3> Cl (£ < .05). 

A supplementary analysis was made for the high, medium, and low 
IQ levels using all five treatment groups having posttests. On all 
three measures of creative problem solving significant differences were 
found in favor of one of the experimental treatments. Usually the favored 
trt xtment was the one with the emphasis on abstract thinking or the one 
with the extra practice on problem-solving concepts. See Table 4 and 
Table 5. 



Another supplementary analysis was made using the intact class as the 
unit of observation (N = 24 class means). For the four treatment groups 
the only significant difference, with this loss of statistical power, 
was for the oral composition. On OCOM, £ < .05 (F ■ 4.11, df = 3/20. 

The mean for the experimental group with the abstract emphasis (E3) > the 
control (Cl; £ <. .05), according to Scheffd's test. Supplementary analy- 
ses were also made using analysis of covariance (IQ and IQ and pretest) 
and results were essentially the same as all of the results for the 
analyses of variance. At the time of the present writing it is not pos- 
sible, however, to use Scheffd test after analyses of covariance, accord- 
ing to communication with Scheffd. 



Discus sion 

The hypothesis of no treatment differences on the criteria of 
problem solving was rejected. Supplementary analysis by moans of Scheff<? 
tests indicated that the group trained not only in Che core program of 
problem solving but also on the subability of qualitative levels of think- 
ing with emphasis on abstract thought (E3) had the highest mean for two 
out of the three problem-solving tests. These tests were the oral com- 
position and the open-ended stories with 12 stimulus questions. The sig- 
nificant contrasts against the control group was consistently in favor of 
the experimental group. However, on the open-ended stories the group 
with the abstract training was also significantly higher than the group 
with the listening emphasis. 

It might be implied that in the case of the written composition, 
the advantage naturally lay with a problem-solving treatment which 
simply included extra practice. All experimental groups which all had 
a basic program in problem solving produced higher means than did the 
control group with no systematic training. 
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Table 4 

Analyses of Variance for Scores on the Posttcst Measure of Creative 
Problem Solving According to Three Levels of Mental Ability 
Utilizing All Five Treatment Groups 



Criterion 

and Level Source df MS F Schcff 6 Test 



WRIC 


Hi It) 


Treatment 


(111-146) 


Error 


Medium IQ 


Treatment 


(96-110) 


Error 


Low IQ 


Treatment 


(62-95) 


Error 


OCOM 


Hi IQ 


Treatment 

Error 


Medium IQ 


Treatment 

Error 


Low IQ 


Treatment 

Error 


OPST 


Hi IQ 


Treatment 

Error 


Medium IQ 


Treatment 

Error 


Low IQ 


Treatment 

Error 



4 


593.1 


5 . 8 *** 


61 


101.5 




4 


109.0 


2 . 6 ** 


61 


42.1 




4 


53.7 


1.7 


63 


32.3 





4 


526.5 


6 , 6 *** 


61 


79.6 




4 


446.0 


3.4*** 


61 


130.7 




4 


263.8 


3.3** 


63 


79.9 





4 


20453.9 


2.3 


61 


8956.9 




4 


31878.9 


3.8*** 


61 


8319.8 




4 


33021.0 


4 , 9 *** 


63 


6740.3 





El® > E2 b **,E3 C * 
Cl d ***,C2 c ** 



El > Cl*** 
E3 ? Cl*** 
E2 > Cl** 

E3 ^ Cl** 



E3 ? Cl* 

E3 ;> E2** 

E3> C2** 

El .? Cl*, El > C2** 
E3p> C2* 



* £ <^.10 (used in connection with Scheff£ tests only) 

** £ .05 

*** £ C.,. 01 

a El = extra problem-solving practice 
b E2 = listening emphasis 
c E3 = abstract thinking emphasis 
<1 Cl = control with pretest 
e C2 - control with posttest only 



O 
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Table 5 

Means for PosttesCs for Three Tests of Creative Problem Solving 
For All Five Treatment Groups at Three IQ Levels 



Test and 
IQ Level 


El 

M 


E2 

M 


E3 

M 


Cl 

M 


C2 

M 


WRIC 


(N * 15) 


(N - 14) 


(N » 17) 


(N - 12) 


(N - 8) ' 


Hi 


70.3 


"57.4 


"59.6 


53.9 


54.4 


Med 


(N » 13) 
"56.5 


(N “ 16) 
"56.9 


(N *» 7) 
57.1 


(N ■ 14) 
55.3 


(N • 16) 
50.6 


Lo 


(N “ 13) 
"52.7 


(N “ 15) 
"51.5 


(N « 15) 
"49.9 


(N - 11) 
47.1 


(N - 14) 
"49.8 


OCOM 


Hi 


62.7 


55.8 


62.4 


48.3 


52.1 


Med 


52.2 


58.2 


60.0 


50.7 


45.3 


Lo 


52.7 


52.0 


56.1 


44.3 


48.1 


OPST 


Hi 


375.5 


353.9 


407.4 


304. i 


343.1 


Med 


322.0 


255.3 


399.1 


307.4 


264.4 


Lo 


290.3 


223.5 


270.9 


190.0 


175.5 



O 
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The finding with respect to the treatment with training in abstract 
thinking corresponds to opinioni data, and recommendations for training 
reported, for example, by Pruner (1964) and Levi (1966). A description 
of the tests of abstract thinking and listening is given elsewhere 
(Lmdsteen and Michael, 1966), 

It may be supposed that further attention to test construction 
and curriculum building in the general area of problem solving and in 
the particular area of assisting the abstract thinking subability may 
aid development of pupil skills. It is recommended that training be 
investigated at beginning grade levels as well as the fifth-grade level 
used in the present study. 
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Table 6 

ANALYSIS OF DIFFICULTY LEVEL OF TWELVE QUESTION TYPES FROM THE 
PROBLEM-SOLVING MEASURE, OPST FOR PRETEST AND POSTEST 
EXPERIMENTAL GROUP ONLY 



Low IQ N - 60 
High IQ N - 65 



IQ Group 


Question No. 


Pretest i 
M 


Postest2 

M 


Rankj 


Rank2 


Low 


i 


20 . 13 a 


25 . 28 a 


1 


1 




2 


17.97 


25.18 


3 


2 




3 


15.33 


18.53 


5 


10 




4 


14.93 


21 .80 


6 


5 




5 


18.43 


24.36 


2 


4 




6 


14.37 


19.74 


9 


7 




7 


1 1 . 97 b 


16 . 79 b 


12 


12 




8 


14.87 


21.67 


8 


6 




9 


13.53 


17.90 


10 


11 




10 


14.93 


19.64 


7 


8 




11 


12.70 


19.21 


11 


9 




12 


16.43 


24.92 


4 


3 


High 


1 


24.19 


37.28 


2 


3 




2 


23.72 


37.16 


3 


4 




3 


19.65 


28.53 


7 


10 




4 


20.12 


33.47 


5 


5 




5 


23.51 


38.94 


4 


2 




6 


19.42 


33.38 


8 


7 




7 


14 . 95 b 


21.94 b 


12 


12 




8 


20.03 


33.41 


6 


6 




9 


17.45 


27.19 


11 


11 




10 


18.95 


31.44 


9 


9 




11 


18.34 


32.84 


10 


8 




12 


24 . 55 a 


43.66 a 


1 


1 



a The questions for the problem-solving measure (OPST) were ranked 
from highest to lowest according to the mean score that high IQ and low 
IQ pupils in the experimental group were able to achieve. For pupils 
in the low IQ division, the highest mean score was made on question 1 
(constructing a main problem) on both the pretest and the postest. For 
pupils in the high IQ division, however, the highest mean was made on 
question 12 (planned evaluation of hypotheses) on both the pretest and 
postest. See Chart 1 for a description of the 12 question types. 
Comparison of pre- and postest ranking of all 12 question types, from 
highest mean score to lowest mean score for the low IQ group, yielded 
a rank-difference coefficient of correlation of .84, and for the high 
IQ group, .93. 

b For pupils in the low IQ division and in the high IQ division 
the lowest mean score was made on question 7 (classification of facts 
and conditions) on both the pretest and the postest. 

ERIC 
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APPENDIX 



Practice on scoring the task. Tell the Problem and Make a Plan. Lot's 
try a practice exercise to see if you can score this fifth-grade child's 
answers to questions concerning an unfinished problem story. Use your 
cover sheet so that you will not see our judgment until you have had 
an opportunity to try to construct your own. Review the steps and the 
qualitative rating of steps within the scale for Tell the Problem and 
Maks a Plan located on page 19 before you try to judge. 

Please mark the appropriate box with an "X." We had better give you 
at least an abbreviated version of the first story (or you nay prefer 
to read the entire story located in the appendix). Dotted lines ... 
indicate omissions. 



Tell the Problem and Make a Plan 

Version C, Openendcd 
Story 1 

Two boys were huddled in the comer of their classroom, heads bent low 
over their science project--an incubator. Finally Jerry speaks. 

"Say, do you think something can be the matter? Our chicken eggs 
should have hatched by now, and they haven't." 

"Well, for Pete's sake don't tell my cousin Tommy," said Bob. 



O 
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"Why not?" 

"You know the way he 'fixed' the volcano we made in class!" 

"Oh, yeah, it wasn't erupting smoke, so he held a match to it. 

There was plenty of smoke then all right--nothing but smoke. ... 

"He should be locked up." 

"jv’o, he's a good kid, most of the time. I really think he's a 
smart kid. Ha never sits still though. ... 

"Back to ovr chicken eggs. It may be that the temperature is^ 
still too low, even though we put in that extra volt light bulb. 

"Or it may be it's too hot now," St.id Bob. "We need to rig up 
some ventilating system, a small electric fan might do it. ... 

"Oh, no I Here comts Tommy 1" said Bob. 

"Hi fellas. - - Hi fellas! - - HI FELLAS ! 1" 

"Hello, Tommy," both boys finally replied. 

"Oh, it's your chicken eggs - - haven't hatched yet, huh? I 
know what to do. /There's a hammer hereJ7 I know just what to do!" ... 



Here is a judgmental scale for you to use. 



Here is the first question which is followed by a fifth-grade 
child's written response to that question. 

Question 1. (Facts and conditions) 

We are going to put our attention on Tommy, the one the boys were 
talking about, the child who had the hammer. You may have some idea 
that Tommy has a problem. If he has a problem list the pieces of 
information in the story that might be helpful to Tommy in solving 
his problem. Write as many pieces of information that you notice as 
you can. Write the facts in the story that 'might work to help Tommy. 



Child's response: 

Question 1 . j think ^omc hoJtp^ut things I noticed wcac that ho 6 cooi-tn 
laid he. wcw Knotty a. good tmaAt kid. He didn't want to lock him up., 
Jenny hod on idea to get Towny to itop ond think fiin&t, ond Bob 6oi o 
h& should tny that... and even etanted to 6ay "ioe can http you ttann... 
That' 6 not much but I gum it'i bttten than nothing. 

(Use a cover sheet for the bottom half of page 20.) 

ERIC 
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Figure 8. 

JUDCE'S SCALE AMD SCORING DEVICE 
for the measure 

TELL THE PROBLEM AND MAKE A PLAN 



Check onej. 
Protest D 
Postest □ 



Put an "X" iii the appropriate box if the quality being rated is present. 
For each "X" given, count two points. Enter the total score in the boxes 
for each question in the column at the end where it says QUESTION TOTAL. 
Even if the response is incomprehensible or the child did not respond at 
all, give one point in total so that there aro no zero scores. 



Quality 


Questions for Story #1 


i 


2 


! 3 


4 


5 


i 6 


7! 81 9 


10 


11 


12 


1. LITERAL COMPREHENSION: response 

shows question understood 














l 


I 




6 th 
c 


gr 

niy 


ade 


2. RELEVANCE: response shows a con- . 

ncction to story; "sticks to topid 1 




i 


1 

i 




















3. ELABORATION: response shows flume} 

and details or describes 


























4. ORIGINALITY: response is unique or 

unusual in the group 


























5. MULTIPLE ALTERNATIVES: response 
shows understanding of several 
alternatives 
















i 










6, EMPATHY; response shows child put-' 
ting himself in another’s place 
or role 


















* 








7. CAUSAL THINKING: response tells 
why or for what reason events 
may have happened 


























8. TESTABLE HYPOTHESES; response 
possible to assess, observe; 
usable, feasible 
































































| 






















L 






?. NUMBER OF HYPOTHESES (write number 
and add in as part of score) 




































- 








i 






















! 






.0, EVALUATION: response shows judg- 
mental effort to evaluate hypo- 
theses with at least implied 
criteria and with supporting 
reasons or evaluative thinking 
on other steps 


















t 

| 




h; 




QUESTION TOTAL 










1 

1 

l 















Pupil Namc_ Code # Story I Total 1 I 



Teacher Name Code # Judge Name, 

,c ->aces lined out indicate ' that quality does not apply) 
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Using your Judge's Seale and Scoring Device for "Tell the Pro -lorn 

and Make a Plan," how would you rate this answer to that question (page 

number 1 on facts and con- 
ditions? Put an "X" (or 
a "2") in the boxes if 
the quality was present. 
Then enter the question 
total in the box at the • 
bottom. Each quality 
checked gets 2 points. 
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Quality 


Question 1 


1. Literal comprehension 




2. Relevance 




3. Elaboration 




A. Originality 




5. Multiple alternatives 




6 • Empathy 




7. Causal thinking 
• • • 




10. Evaluation 




Question Total 





(Qualities 8 and 9 do 
not apply to this ques- 
tion or the next ones 
until you get to ques- 
tion #8.) 



STOP until finished marking the boxes. Thank you. 



Answer: Here is the way we scored. You may or may not agree. 



O 
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Quality 


Question 1 


1. Literal comprehension 


X 


2. Relevance 


X 


3. Elaboration 


X 


A. Originality 




5. Multiple alternatives 


X 


6. Empathy 




7. Causal thinking 
• • • 




10. Evaluation 


X 


Question Total 


10 



tcntly gave the response the benefit of the 
in ( ) refer to the scale.) 



We felt that for this ques- 
tion-step that the response 
showed (1) comprehension 
of what the question was 
asking for, (2) that the 
response was relevant to 
the story material given 
as well as to the question, 
(3) that the response 
showed several facts and 
conditions giving elabor- 
ation as well as (5) mul- 
tiple alternatives, and, 
although we did not feel 
very strongly about this, 
we decided that the last 
sentence showed some (10) 
evaluative thinking. As 
each of the 5X's was worth 
2 points, the total for the 
question was 10. W5v: n there 
was uncertainty we consis* 
doubt, or a mark. (Numbers 



(Use a cover sheet for bottom half, next page.) 
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Please go to the next question and response. 

You may wish to transfer your scoring to page • 19 into the first 

column. 



Here is the second question which is followed by a child's written 
response to that question based on the same story. 

Question 2, (Definition) 

Here is an idea that is related to that story. The idea is "prob- 
lem solving." How would you define or tell the meaning of the words 
"problem solving"? Write your answer. 



Child's response: 

Question 2. I think it mate hou you think you zouid itop uhat you m ant to 
t>top, on to uioak out an aaithmztic pKobte.'n, ok a tnoubte. bitwttn ^niindb. 



Using your Judge's Scale and Scoring Device for "Tell the Problem 
and Make a Plan''given you on page 19 » mark the boxes if you feel 
the quality is present in the child's response. Be sure you mark in 
the second column for question 2. 

STOP until finished marking the boxes on page 19 column 2. Thank 
you. 



Answer: Here is the way we scored. You may or may not agree. 



Quality 


Question 2 


1. Literal comprehension 


X 


2, Relevance 


X 


3. Elaboration 


X 


4. Originality 




5. Multiple alternatives 


X 


6, Empathy 




?. Causal thinking 
• • • 




10, Evaluation 




^Question Total 


3 



We felt that for this ques • 
tion or step that the res- 
ponse showed (1) compre- 
hension of what the ques- 
tion required, (2) that 
the response was relevant 
to the question and to the 
story context. Because of 
examples "arithmetic prob- 
lem" and "trouble between 
a friend," we marked the 
quality of (3) elaboration. 
And because alternative 
meanings of "problem solving" 
were implied we marked the 
box for (5) multiple alter- 
natives. Thus on this ques- 
tion the response earned a 
score of 3. (2 points times 

4 qualities • 6) 



I 
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Please go Co the next question and response. (Use cover sheet for 
bottom half of this page.) 



Here is the third question which is followed by a child's written 
response based on the same story. 

Question 3. (Subproblems) 

What arc some of the smaller problems or parts of problems that 
Tommy has in this story? Fcnomber we are thinking about Tommy. You 
can look back at the story if you need to. Write your answer. 



Child's response: 

Question 3. I think iomz o£ hi & ifnalZ pAobli/ni u'ZAZ ti^at hz had 1 

buA/izd up a volcano, and I bzt hz got In tnoublz about thz c oX and 2 

thz HJUU.U, cold Jzaau dozin' t izzm to Ukz hi>n, and thz boy i didn't 3 
mnt to tzUL hon anything ok talk to him. Hz may \ \zzl bad and not 4 
undzn&tnnd and no onz tzlli him. Hz doz&n't do thengi thz imaht mxjt 5 
hz dozt> it thz nutty my. 6 



Using your 10 point Juuge's Scale and Scoring Device on page 
19 mark the box for each quality you feel is present in the 
child';, response. Be sure to mark in the third column for question 3. 

STOP until finished marking the boxes. Thank you. 




Answer: Here is the way we scored. We 



Quality 


■■s' ' ■ a^aa 

Question 3 


1. Literal comprehension 


X 


2. Relevance 


X 


3. Elaboration 


X 


4. Originality 


X 


5. Multiple alternatives 


X 


6. Er.pathy 


X 


7. Causal thinking 
* • » 




10. Evaluation 




Question Total 


12 



the 



felt that the response showed 

(1) comprehension of what 
the question required, 
that the response was (2) 
relevant to the question 
and to the story context. 
We felt that there were 
details for (3) elabora- 
tion rather than a flat 
statement. We might have 
trouble defending this, but 
in the group of papers 
we judged, we felt that 
the last sentence was a 
rather (4) original or 
unique way to express the 
subprobloia (line 6). In 
line 4 and 5 we thought 
we detected (6) empathy 
or an understanding of how 
Tommy might feel. Conse- 
quently, on this question 
response earned a score of 12. 



